FCN-Transformer Feature Fusion for Polyp Segmentation

Abstract Colonoscopy is widely recognised as the gold standard procedure for early detection of colorectal cancer (CRC). Segmentation valuable two significant clinical applications, namely lesion and classification, providing means to improve accuracy robustness. The manual segmentation polyps in colonoscopy images time-consuming. As a result, use deep learning (DL) automation polyp has become important. However, DL-based solutions can be vulnerable overfitting resulting inability generalise captured by different colonoscopes. Recent transformer-based architectures semantic both achieve higher performance better than alternatives, however typically predict map $$\frac{h}{4}\times \frac{w}{4}$$ h 4 × w spatial dimensions $$h\times w$$ input image. To this end, we propose new architecture full-size which leverages strengths transformer extracting most important features primary branch, while compensating its limitations prediction with secondary fully convolutional branch. from branches are then fused final map. We demonstrate our method’s state-of-the-art respect mDice, mIoU, mPrecision, mRecall metrics, on Kvasir-SEG CVC-ClinicDB dataset benchmarks. Additionally, train model each these datasets evaluate other superior generalisation performance. Code available: https://github.com/CVML-UCLan/FCBFormer .